• Class begins and ends on time
• Listen to each other with respect and trust
• Invest in your own learning
• Participate actively in your learning

• Identify some of the tools psychologists use to conduct research and communicate findings
• Define what you want to learn

• Class structure
• What makes a study convincing?
• The Scientific Method
• Transparent Science

• Each class will usually consist of:
  • lecture
  • discussion
  • group/individual work
  • maybe a video

• No textbook!
• But I recommend:
  • R for Data Science
  • Writing Science by Joshua Schimel
  • Ten simple rules for structuring paper by PLOS Computational Biology Staff
  • Writing for Psychology by Mark Mitchell and colleagues

• At the end of this course you will be able to:
  • Critically evaluate social psychological research.
  • Integrate your unique hypotheses with relevant published research.
  • Design and build an experiment using modern tools.
  • Recruit participants while protecting their rights as human subjects.
  • Use data analysis software to evaluate social psychological research outcomes.
  • Report your research results in an oral presentation and in written form in accordance with APA guidelines.

• low p-value
• lots of participants
• probability sample
• preregistration
• replication
• can check the data myself

• When an assumption is violated

• A p-value represents how surprising the data is, assuming the null hypothesis is true

• A p-value represents how surprising the data is, assuming the null hypothesis is true
• Null hypothesis: Harry is not a wizard
• Alternative hypothesis: Harry is a wizard
• Data: Hagrid told Harry he is a wizard.
• Should we reject the null hypothesis? How surprising is this data, assuming Harry is not a wizard.

• A p-value represents how surprising the data is, assuming the null hypothesis is true
• A p-value is significant when it is less than .05
• In your mind, switch the word ‘significant’ to ‘surprising, assuming the null hypothesis is true’
• A p-value is the probability of seeing data as or more extreme than the current data, assuming the null hypothesis is true.

• Does this mean there is a 95% probability the alternative hypothesis is true?

• A p-value represents how surprising the data is, assuming the null hypothesis is true
• Scientists use an alpha level to decide whether to reject the null hypothesis. It is usually set to p < .05.
• This does not mean there is a 95% chance the alternative hypothesis is correct.
• In fact, we never know if we were correct in rejecting the null hypothesis.
• We just know that in the long run if we set our alpha level to p < .05, we’ll never be wrong more than 5% of the time.

• Or ask your phone to

• The magnitude of an effect

• Are women better coders than men?
• Break into groups of 4-6
• Write your names and n-numbers on a piece of paper
• Women are better coders than men: p < .05, d = .04
• Imagine a man and a woman have applied to be a coder for your business. Would you automatically hire the woman? How much more likely would you want a randomly selected woman to be better at coding before you would use gender as a basis for hiring a coder?
• Once your group has reached consensus, hand in your answers to the three questions.

• Assuming the null hypothesis is false, what is the probability the p-value will be less than .05?

• Assuming the null hypothesis is false, what is the probability the p-value will be less than .05?
• related to effect size and sample size
• Statistical power increases when you have a larger sample
• Statistical power increases when you have a larger effect
• Rule of thumb is to aim for 80% power

• To answer this we need to know:
  • How many tests conducted
  • How many participants

QRP

Questionable Research Practice

• Almost 1/4 of American Psycholgists have used a QRP in the last 12 months (https://psyarxiv.com/3v7hx/)

HARKING

Hypothesizing after results known

p-hacking

another phrase to describe a questionable research practice but usually with a focus on obtaining a significant p-value

• Register details of a study beforehand so you don’t exercise researcher degrees of freedom later
• Design
• Sample size/Stopping rule
• Analysis Plan and exclusion rules

• The extent to which results of a study can be generalized to other populations and settings
• Exact replication: generalize to other labs
• Conceptual replication: generalize to other operationalizations of variables
• Other populations
• Other experimenters
• Replication plus extension: researchers replicate original study but add variables to test additional questions

Replication

If I re-run your experiment, do I get the same results?

• About 35% of Social Psychology experiments replicate

Reproduction

If I re-analyze your data, do I get the same results?

In the journal cognition:

• About 78% of data is available to be re-analyzed
• About 60% of data is actually usable
• Of 35 articles selected for a test:
  • 11 could be reproduced
  • 11 more could be reproduced with help from the authors
  • 13 articles not be reproduced even with author help

• http://curatescience.org

• Willpower is like a muscle. It gets tired as you use it. So you have less willpower and doing a task that requires willpower.
• Haggar et al., 2010: Meta-analysis of 83 studies finds evidence for ego depletion (d = .63—medium)
• Carter & McCullough (2014): Another meta-analysis but correcting for small study effects (d = 0)
• Haggar et al., 2016: Many labs study (23 labs), 2141 participants. Ego depletion is a small effect (d ~= 0).

• Benefits:
  • This is what people think science is anyway
  • make science more democratic and more meritocratic
• Solves:
  • More people can be involved. More collaboration, more sharing of data and materials
  • helps science to self-correct
• Cons:
  • Takes more time initially
  • (but saves time in the long run by making data and materials available)

• low p-value?
• No QRPs, p-hacking, or HARKING
• open science practices
• replication
• reproducibility
• construct, internal, external, and statistical validity

• No Class!
  • (I won't be here)
• Submit assignments on Canvas
• Take the quiz!
• Check Canvas for updates
• TAKE THIS SURVEY
  • https://unf.co1.qualtrics.com/jfe/form/SV_0riDh46f9IBc6yx